In the latest in our series of interviews meeting the AAAI/SIGAI Doctoral Consortium participants, we caught up with Aniket Roy to find out more about his research on generative models for computer vision tasks. Tell us a bit about your PhD - where did you study, and what was the topic of your research? I recently completed my PhD in Computer Science at Johns Hopkins University, where I worked under the supervision of Bloomberg Distinguished Professor Rama Chellappa. My research primarily focused on developing methods for resource-constrained image generation and visual understanding. In particular, I explored how modern generative models can be adapted to operate efficiently while maintaining strong performance.

We present an architecture which lets us train deep, directed generative models with many layers of latent variables. We include deterministic paths between all latent variables and the generated output, and provide a richer set of connections between computations for inference and generation, which enables more effective communication of information throughout the model during training. To improve performance on natural images, we incorporate a lightweight autoregressive model in the reconstruction distribution. These techniques permit end-to-end training of models with 10+ layers of latent variables. Experiments show that our approach achieves state-of-the-art performance on standard image modelling benchmarks, can expose latent class structure in the absence of label information, and can provide convincing imputations of occluded regions in natural images.

Neural Information Processing Systems http://nips.cc/

Our algorithm is few-shot: Given about a dozen images restored by the model, it can significantly improve the perceptual quality and/or the MSE of the model for newly restored images without further training.

Unrestricted adversarial attacks typically manipulate the semantic content of an image ( e.g., color or texture) to create adversarial examples that are both effective and photorealistic, demonstrating their ability to deceive human perception

Developing amethod forhigh-quality reconstruction ofseenimages fromthecorresponding brain activity is an important milestone towards decoding the contents of dreams and mental imagery (Fig 1a). In this task, one attempts to solve for the mapping between fMRI recordings and their corresponding natural images, using many "labeled"{Image, fMRI} pairs (i.e., images and their corresponding fMRIresponses).